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@ Image processing apparatus. 

@ An image processing apparatus having a sol- 
Id state image sensor for inputting an optical 
image thereto and converting the image into an 
electrical signal ; an image memory having a 
general-purpose DRAM formed by at least one 
chip and required to control the refresh oper- 
ation from the outside of the memory chip; a 
timing signal generator for generating timing of 
read from the solid state image sensor ; a trans- 
fer/storage control device for horizontally read- 
ing data from the solid state image sensor 
synchronously with the horizontal read timing 
signal generated from the timing signal 
generator, transferring the read data and stor- 
ing the data in the DRAM ; a first refresh signal 
generator for refreshing the DRAM at predeter- 
mined time intervals when no data is read from 
the solid state image sensor; and a second 
refresh signal generator tor refreshing the 
DRAM a predetermined number of times during 
the dormant period of the horizontal read tim- 
ing signal. In the apparatus, the second refresh 
signal generator Is made effective synchron- 
ously with the horizontal read timing signal 
during the time of read from the solid state 
image sensor, and the first refresh signal 
generator Is made effective during a time other 
than the time of read from the solid state image 
sensor. 
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The present invention relates to an image proc- 
essing apparatus such as an electronic camera or the 
like which is configured so as to electronically record 
i mage Information, and particularly to an Image proc- 
essing apparatus in a memory control system in 5 
which t he output data from a solid state image sensor 
is stored in a DRAM. 

A conventional electronic camera corresponding 
to a television system (NTSC or PAL) frequently uses 
SRAM as memory for storing the data of a solid state 
image sensor because of high access speed. The 
SRAM has the disadvantage that a memory capacity 
which is only enough for one image can be contained 
in the camera body because of the small capacity and 
high cost of the SRAM. However, the SRAM has ad- 
vantages with respect to simple control of writes and 
reads, rapid access, and a relatively short time re- 
quired for transferring data because the data capaci- 
ty handled in the camera is small, as described 
above. The execution of writes and reads to and from 
memory and the memory address management ac- 
companying the execution can be controlled directly 
by a microprocessor, without the need for a special 
memory control system. However, when a recent 
HDTV system (High Def inition Television) is used, the 
memory capacity must be increased as the amount of 
the pixel data for one image is increased, and reads, 
writes and transfers of the data must be performed at 
highspeed. It is preferable from the viewpoint of a bal- 
ance of cost and packaging area to use a general pur- 
pose DRAM memory for an electronic camera which 
requires a large amount of data. 

However, when DRAM is used as memory, read 
from a solid state image sensor must be temporarily 
(refresh cycle) stopped for a refresh operation of the 
DRAM. The time taken from charge storage in the 
solid state Image sensor to data read therefrom is in- 
creased by the stop time, thereby generating much 
dark current and causing noise spots on the sensor. 

The control of the DRAM is also complicated, and 
the processing speed thereof is lower than that of the 
SRAM. In addition, the refresh operation is required 
for holding the stored contents, conditions for ad- 
dress setting and write and read timing are severe, 
and the access speed is low. A system is thus re- 
quired for appropriately controlling the DRAM while 
relating it to the solid state Image sensor and other 
peripheral devices. 

A solid state image sensor for color images gen- 
erafly has color filters having a plurality of colors 
(RGB orYMC) which are bonded to the surface there- 
of in a mosaic or stripe form so that a color is repre- 
sented by combination of output of a plurality of pixels 
corresponding to the filters. If the output of a pixel is 
saturated, therefore, the color of the periphery of the 
pixel cannot be correctly reproduced. However, when 
a general object is photographed, the output of the 
pixels in a port ion or t he whole of the i mage plane of 



the solid state image sensor are sometimes saturat- 
ed because the object Is excessively bright. 

If the output from a pixel is greater than a prede- 
termined value, it is generally decided that the pixel 
is saturated. In reproduction, the gain (GAIN) of the 
color of the periphery of the pixel is thus decreased 
by signal processing for preventing the occurrence of 
a false color. 

However, when t he dark current of the solid state 
Image sensor is decreased for decreasing the noise 
thereof on the basis of the image pickup data of the 
object, as in a conventional element, since the dark 
currents of the respective pixels are nonuniform, the 
level of a pixel which is saturated is decreased due to 
subtraction. As a result, ft is impossible to discrimin- 
ate an unsaturated pixel and a saturated pixel. There 
is thus the problem of difficulty in processing for sup- 
pressing a false color of a high-brightness portion. 

The decrease in dark current also causes the fol- 
lowing problem: When general recorded data is repro- 
duced to an image, a high noise component is de- 
creased by passing through a filter. In this case, it is 
assumed that there is a dark current noise, as shown 
in Fig. 14(b) below. In Fig. 14, the horizontal position 
is shown on the abscissa, and the amplitude is shown 
on the ordinate. In this case, the result of subtraction 
of the dark current as shown in Fig. 14(b) from origi- 
nal data as shown in Fig. 14 (a) is as shown in Fig. 
14(c). The resultant data is passed through a filter to 
produce data as shown in Fig. 14(d). There is thus the 
problem that the noise cannot completely be re- 
moved. 

In consideration of the above problems of con- 
ventional apparatuses, in af irst aspect the invention 
aims to provide an image processing apparatus 
which can obtain a good image without the remark- 
able unevenness caused by differences in the stor- 
age time of a solid state image sensor. 

In an alternative aspect, the invention aims to 
provide an image processing apparatus which can 
simply realize reproduction processing for suppress- 
ing the color of a high-brightness portion. 

In another alternative aspect, the invention aims 
to provide an image processing apparatus which can 
effectively utilize a memory capacity. 

In accordance with a first embodiment of the 
present invention, an Image processing apparatus 
comprises a solid state image sensor for inputting an 
optical image thereto and converting the input image 
into an electric signal; image memory comprising a 
general purpose DRAM formed by at least one chip 
and having the need for controlling refresh opera- 
tions from the outside of the memory chip, a timing 
signal generator for generating a timing signal for 
reading from the solid state image sensor; trans- 
for/storage control means for horizontally reading 
data from the solid state image sensor synchronously 
with the horizontal read timing signal generated from 
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the timing signal generator, transferring the read 
data to the DRAM and storing the data therein; first 
refresh signal generation means for refreshing the 
DRAM at predetermined time intervals when no data 
is read from the solid state i mage sensor and second 5 
refresh signal generation means for refreshing the 
DRAM a predetermined number of times during the 
dormant period of the horizontal read timing signal. In 
the apparatus, the second refresh signal generation 
means is made effective synchronously with the hor- 10 
izontal read timing signal during read from the solid 
state image sensor, and the first refresh signal gen- 
eration means is made effective during the time 
other than the time of read from the solid state image 
sensor. 

In accordance with a second embodiment of the 
present invention, an image processing apparatus 
comprises a solid state image sensor for inputting an 
optical image thereto and converting the image into 
an electric signal; a storage medium for A/D convert- 
ing the output data from the solid state image sensor 
and storing the converted data therein; means for 
storing the output of the A/D conversion of a first im- 
age from the solid state image sensor in a first pre- 
determined region of the storage medium; means for 
storing t he output of the A/D conversion of a second 
image from the solid state image sensor in a second 
predetermined region of the storage medium; and op- 
eration storage means for adding a first predeter- 
mined value to the content of the second storage re- 
gion of the storage medium, subtracting the content 
of the first storage region of the storage medium from 
the result of the addition, and storing the result of the 
subtraction in the second region of the storage me- 
dium. When the output value of the A/D conversion of 
any desired pixel of the solid state image sensor in 
which the second image is stored is within a first pre- 
determined range, a second predetermined value is 
stored in a storage region of the storage medium cor- 
responding to the pixel. When the result value of ad- 
dition is within a second predetermined range, the re- 
sult value is replaced by a third predetermined value. 
When the result value of subtraction is within a third 
predetermined range, the result value is replaced by 
a fourth predetermined value. 

In accordance with a third embodiment of the 
present invention, an image processing apparatus 
comprises a solid state image sensor for inputting an 
optical image thereto and converting the image into 
an electric signal; a memory for storing the output 
data from the solid state image sensor; first, second, 
third and fourth registers for setting first, second, 
third and fourth addresses, respectively, for writing 
data in the memory; first counter means initialized to 
the contents of the first address; first comparison 
means for comparing the contents of the first counter 
means with the contents of the second register; sec- 
ond counter means initialized to the contents of the 



third address; second comparison means for compar- 
ing the contents of the second counter with the con- 
tents of the fourth register address selection means 
for select ing either of the contents of the first cou nter 
and the contents of the second counter to output the 
selected contents to the memory; and a memory con- 
trol circuit for controlling the operation of the mem- 
ory. 

In accordance with a fourth embodiment of the 
present invention, the first address is a start address 
for writing data in the memory, the second address is 
an ending address for writ ing data in the memory, the 
third address is a start address for reading data from 
the memory, and the fourth address is an ending ad- 
dress for reading data from the memory, data fetch 
from the solid state Image sensor being stopped on 
the basis of output from the first or second compari- 
son means. 

In accordance with a fifth embodiment of the 
present invention, an image processing apparatus 
further comprises second memory means for storing 
the data contents of the memory; means for contin- 
uously accessing any desired region on the memory; 
and means for transferring in blocks, to the second 
storage means, the data of any desired region on the 
memory having a capacity smaller than that of one 
image plane of the solid state image sensor so as to 
transfer data from the solid state i mage sensor to the 
memory between respective transfer data blocks. 

In accordance with the first embodiment, the re- 
fresh operation of the DRAM is performed in the dor- 
mant period (for example, horizontal blanking period) 
of the horizontal read signal, whereby reading from 
the solid state Image sensor is not stopped in the 
course of reading an image. 

In accordance with the second embodiment, 
when the output value of A/D conversion of any de- 
sired pixel of the solid state image sensor in which a 
second image (for example, a dark current pattern) 
corresponding to a first i mage (for example, an object 
image) is within a first predetermined range, a sec- 
ond predetermined value is stored in a storage region 
on the storage medium corresponding to the pixel, 
and when the value resulting from addition of the first 
predetermined value to the contents of the second 
storage region of the storage medium is within a sec- 
ond predetermined region, the addition result is re- 
placed by a third predetermined value. When the val- 
ue resulting from subtraction of the contents of the 
first storage region from the addition result is within 
in a third predetermined range, the addition result is 
replaced by a fourth predetermined value. Even if 
noise enters the dark current image, the noise is t hus 
offset, and a dark current can be subtracted without 
an adverse effect on an original image. 

In accordance with the third to fifth embodi- 
ments, an image processing apparatus further com- 
prises a dedicated memory controller provided in a 
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camera, and at least four registers for setting ad- 
dresses on the memory and various sequencers for 
writing and reading, which are provided in the control- 
ler, whereby the DRAM can appropriately be control- 
led while being related to the solid state image sensor 5 
and other peripheral devices. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram illustrating an electronic 10 
camera in accordance with a first embodiment of 
the present invention; 

Fig. 2 is a block diagram illustrating the memory 
controller shown in Fig. 1; 

Fig. 3 is a block diagram continued from Fig. 2; 1 s 
Fig. 4 is a drawing illustrating data read from a 
solid state image sensor 
Fig. 5 is another drawing illustrating data read 
from the solid state image sensor 
Fig. 6 is a drawing illustrating a circuit related to 20 
refresh timing control; 

Fig. 7 is a drawing illustrating the configuration 
of a DRAM of an electronic camera in accordance 
with a second embodiment of the present inven- 
tion; 25 
Fig. 8 is a drawing illustrating the address space 
of the DRAM in the second embodiment; 
Fig. 9 is a drawing illustrating the operation tim- 
ing stored in the DRAM; 

Fig. 1 0 is a flowchart illustrating the operation of 30 
the second embodiment of the present invention; 
Fig. 11 is a flowchart continued from Fig. 10; 
Fig. 12 is a block diagram illustrating details of 
the register 1 1 7, an operation section 11 8 and the 
peripheral portion thereof shown in Fig. 3; 35 
Fig. 13 is a flowchart illustrating the processing 
operation of the second embodiment; 
Figs. 14(a) through 14(g) are drawings illustrat- 
ing the effects of the second embodiment; and 
Fig. 1 5 is a flowchart illustrating the operation of 40 
a third embodiment of the present invention. 

DETAILED DESCRIPTION OF EXEMPLARY 
EMBODIMENTS 

45 

Embodiments of the present invention are de- 
scribed below with reference to the drawings. 

Fig. 1 is a block diagram illustrating the config- 
uration of an image processing apparatus (electronic 
camera) in accordance with a first embodiment of the so 
present invention. 

In Fig. 1, reference numeral 1 denotes an image 
pickup lens; reference numeral 2, alight control mem- 
ber comprising a diaphragm, a shutter and so on; ref- 
erence numeral 3, a solid state image sensor such as 55 
CCD or the like; reference numeral 4 r a sample-and- 
hold circuit for sampling and holding the output from 
the solid state image sensor 3; reference numeral 5, 



an A/D converter; reference numeral 6, a memory 
controller for controlling the refresh operation of 
DRAM, write and read; reference numeral 7, the 
DRAM; reference numeral 8, a timing signal genera- 
tor for generating a timing signal and a sample-and- 
hold pulse for driving the solid state image sensor 3, 
and an A/D conversion pulse; reference numeral 9, a 
control section for controlling a system for an image 
pickup sequence; reference numeral 10, a storage 
medium such as a hard disk; and reference numeral 
11, a trigger switch. 

A description will now be made of a flow of image 
data accompanying photography by a camera. 

An object image passed through the image pick- 
up lens 1 is projected on the solid state image sensor 
3. At this time, the control section 9 controls the light 
control member 2 so that exposure is performed ap- 
propriately. 

The image data stored in the solid state image 
sensor 3 is successively read synchronously with the 
timing signal generated from the timing signal gen- 
erator 8. The data is then sampled and held by the 
sampie-and-hold circuit 4, and converted to a digital 
value by the A/D converter 5. The A/D converted data 
is stored in the DRAM 7 under control by the memory 
controller 6. The image data stored in the DRAM 7 Is 
read with predetermined timing by the control sec- 
tion 9, and is transferred to the hard disk 10 to bring 
the data flow to an end. 

The storage of the A/D converted data in the 
DRAM 7 under control by the memory controller 6 in 
accordance with the present invention is described in 
detail below. 

Figs. 2 and 3 are block diagrams of the memory 
controller 6 shown in Fig. 1. 

In Figs. 2 and 3, reference numeral 125 denotes 
a system bus which is connected to the control sec- 
tion 9 shown in Fig. 1 so that a command and data are 
transmitted between the control section 9 and the 
memory controller 6 through the system bus 125. In 
the drawings, reference numerals 101, 102, 103 and 
104 respectively denote DRAM address registers tor 
specifying addresses in the DRAM. The address reg- 
isters 101 and 102 set READ addresses for reading 
data from the DRAM, and the address registers 103 
and 104 set WRITE addresses for writing data to the 
DRAM. 

The address registers 101 and 103 set the start 
addresses which will be described below, and the ad- 
dress registers 102 and 104 set the ending address- 
es. 

Reference numerals 105 and 107 each denote a 
counter for updating an address for accessing the 
DRAM, and reference numerals 106 and 1 08 each de- 
note a comparator for making comparison and deci- 
sion whether or not the present access address 
reaches the ending address. Reference numeral 1 09 
denotes a register for setting a DRAM refresh cycle; 
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reference numeral 114, a refresh counter for counting 
the refresh cycle; reference numeral 110, a register 
for receiving a command given from the control sec- 
tion 9 to the memory controller 6; reference numeral 
115, a command interpreter for interpreting the kind 5 
of the command; reference numeral 111, a WRITE 
data register for holding write data when any desired 
data is written in the DRAM; reference numeral 112, 
a READ data register for transferring the data read 
from the DRAM to the control section 9; reference nu- 10 
meral 113, a constant register for storing correction 
constant data for correcting the data stored in the 
DRAM; reference numeral 1 17, a register or buffer for 
storing data to be written to or read from the DRAM; 
reference numeral 11 8, an operation section for com- is 
puting a correction value; reference numeral 119, a 
buffer for inputting data read from the DRAM thereto; 
reference numeral 120,a bufferfor writing data tothe 
DRAM; reference numeral 121, a data width conver- 
ter for rearranging t he data (40 bits) from the DRAM 20 
to the bus width (16 bits) of a hard disk interface; ref- 
erence numeral 122, the hard disk interface for trans- 
mitting data and commands to the hard disk 10; and 
reference numeral 123, an AID data registerfor hold- 
ing the data output from the AID converter and timing 25 
with the Inside of the memory controllers, which reg- 
ister is latched with clock CLK2. Although not shown 
in Figs. 2 and 3 f the whole memory controller 6 is op- 
erated with timing with dock CLK1. Reference nu- 
meral 124 denotes a data bus having a 40 bit width so 
for reading and writing data from and in the DRAM. 
Reference numeral 116 denotes a controller for con- 
trolling the execution timing of the command re- 
ceived by the controller In the memory controller 6 
from the outside and data transfer. Reference numer- 35 
als 126 and 127 each denote a trigger signal for read- 
ing data from the AID converter 5. The trigger signal 
126 is a signal for specifying the start timing of vert- 
ical read from the solid state image sensor 3, which 
is referred to as "VGATE" hereinafter. The trigger sig- 40 
nal 127 is a signal for specifying the start timing of 
horizontal read, which is referred to as "HGATE' here- 
inafter. Reference numeral 128 denotes a line group 
output from the command interpreter 115. 

Reference numeral 129 denotes a refresh re- 45 
quest signal output from the refresh counter 1 14. Ref- 
erence numeral 130 denotes a DRAM address buffer 
for outputting the contents of the counter 105 or 107 
tothe DRAM on the basis of a selection signal output 
from the controller 116. so 

A description will now be made of the operation 
of storing data from the solid stats Image sensor to 
the DRAM in this embodiment configured as descri- 
bed above. 

The addresses of writes to the DRAM are previ- 55 
ously set In the address registers 1 03 and 104 shown 
in Fig. 2. The command (referred to as "PAGE WR 
MODE" hereinafter) to read from the solid state im- 



age sensor is written in the command register 110. 
The timing signal generator 8 shown in Fig. 1 gener- 
ates the signals VGATE and HGATE synchronously 
with the read from the solid state image sensor. This 
is described below with reference to Figs. 4 and 5. 

Rg. 4 is a conceptual drawing of timing of the 
above-described signals VGATE and HGATE 

In Fig. 4. when t he signal VGATE is at a HIGH lev- 
el, data is read from the solid state image sensor. 
When the signal HGATE is at a LOW level, data read 
is stopped. Data for predetermined horizontal lines is 
read from the solid state image sensor during the 
time from rising of the signal HGATE from the LOW 
level to the HIGH level to fall to the LOW level. This 
is described with reference to Fig. 5. The portion ab 
of the signal HGATE shown in Fig. 4 corresponds to 
the portion ab shown in Fig. 5. Likewise, the portion 
cd of the signal HGATE shown in Fig. 4 corresponds 
to the portion cd shown in Fig. 5. The refresh opera- 
tion of the DRAM in accordance with the present in- 
vention is performed when the signal HGATE is LOW. 

The refresh timing control is described below 
with reference to Fig. 6. 

Fig. 6 shows a circuit contained in the controller 
116 shown in Fig. 2. However, Fig. 6 shows only the 
timing control section for the refresh operation which 
is directly related to the present invention, not the 
whole controller 116. Reference numeral 128 denotes 
one of the output signals from the command interpret- 
er 115 shown In Rg. 2. The signal 128 indicates a 
mode for writing data read from the solid state image 
sensor to the DRAM and which is referred to as 
"PAGE WR MODE" (Page Write Mode) hereinafter. 

Reference numerals 200 and 201 each denote a 
NAND gate; and reference numerals 202, 203, 204 
and 205 each denote a INVERTER gate. Reference 
numeral 206 denotes a NOR gate; reference numeral 
207, a DRAM signal generator for generating, to the 
DRAM, various signals such as RAS 209, CAS 
210, /W 21 1 etc; reference numeral 208, a refresh re- 
quest signal generation counter for generating a re- 
fresh request signal in the PAGE WR MODE; refer- 
ence numeral 209, a row address strobe for the 
DRAM; reference numeral 210, a column address 
strobe for the DRAM; reference numeral 21 1 , a write 
strobe; reference numeral 212, a refresh request sig- 
nal in a normal state; reference numeral 213, a re- 
fresh request signal in the PAGE WR MODE; and ref- 
erence numeral 214, a refresh execution signal. 

The operation of the circu it shown In Fig. 6 is de- 
scribed below. 

When no data is read from the solid state image 
sensor, both signals VGATE and HGATE are at the 
LOW level, as described above. At this time, the re- 
fresh request signal is periodically brought into the 
HIGH level by the refresh counter 114 shown in Fig. 
2. The NAND gate 200 thus periodically outputs the 
signal Low, and the refresh execution signal is peri- 
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odically input to the DRAM signal generator 207 
through the NOR gate 206. The DRAM signal gener- 
ator 207 consequently generates the refresh signal. 

The refresh operation may be performed by, for 
example, the CAS BEFORE RAS method. 5 

The DRAM signal generator 207 is a sequencer 
which is designed so as to output the CAS BEFORE 
RAS signal once when receiving the refresh execu- 
tion signal 214. 

Data read from the solid state image sensor is 10 
described below. When data is read from the solid 
state image sensor, both signals VGATE and HGATE 
are as shown in Fig. 4, as described above. Since the 
signal VGATE becomes the HIGH level, the normal 
refresh request signal (SREREQ 21 2) is not generat- 1 $ 
ed because the RFREQ signal 129 is Inhibited by the 
NAND gate 200. 

On the other hand, the gate 201 becomes the 
LOW level during the time the signal HGATE is the 
LOW level (during the period ab shown In Fig. 4). The 20 
refresh request signal generation counter 208 for the 
PAGE WR MODE generates the refresh request sig- 
nal a predetermined number of times, as shown by 
PRFREQ in Fig. 4. In Fig. 4. the refresh request signal 
is generated two times during the time the signal 25 
HGATE is at the LOW level). In this way, when data 
is read from the solid state image sensor, the refresh 
operation id executed during the time the signal 
HGATE is the LOW level. 

Although, in the first embodiment, the numberof so 
refresh operations is two during the time the signal 
HGATE is the LOW level, the number of refresh op- 
erations depends upon the kind of the DRAM, the 
numberof element lines possessed by the solid state 
image sensor, and the time required for reading data 25 
from the elements. 

For example, when the refresh operation is per- 
formed with refresh cycles of 4096 cycles/84 milli- 
seconds, i.e y 4096/64 = 6400 (cycle/sec), by the CAS 
BEFORE RAS method according to the specifics- 40 
tions of the DRAM, and when the solid state image 
sensor has 1024x 1024 pixels and requires 100 micro 
seconds for reading data for one row, since the mint- 
mum cycles for read of one row (1 00 micro seconds) 
are 6.4 cycles, at least 7 cycles of refresh operations 45 
may be performed. The horizontal read ending period 
is of course set to a time sufficient to perform at least 
7 refresh operations. Although write in the DRAM is 
not described above, the write in the DRAM may be 
performed in the high-speed page mode. so 

The number of refresh cycles may be changed so 
as to cope with other types of DRAM and solid state 
image sensors. Since the usual refresh cycles may 
be changed by changing the count period of the coun- 
ter 114 shown in Fig. 2, desired cycle data may be 55 
written in the refresh cycle register 109. 

Similarly, when data is read from the solid state 
image sensor, the counter 208 shown in Fig. 6 may 



be set to a value PRESETABLE. As a result, since the 
PRFREQ (refresh request signal) is output any de- 
sired numberof times, the numberof refresh opera- 
tions during the time the signal HGATE is the LOW 
level can be changed . Although , in the above descrip- 
tion, it is assumed for the sake of simplification that 
data is read, line by line, from the solid state image 
sensor, data for several lines may be read at a time. 
In this case, the solid state image sensor may have 
a plurality of output terminals, and a change-over 
switch may be provided in front of a single A/D con- 
verter. Alternatively, data from the plurality of output 
terminals may be written In the memory as It was. All 
of these cases, of course, are within the scope of the 
present invention. 

An Image processing apparatus in accordance 
with a second embodiment of the present invention 
Is described below. 

Although the second embodiment has the same 
configuration as that of the first embodiment, shown 
in Figs. 1 and 2, it is assumed that the DRAM has the 
configuration shown in Fig. 7. 

Namely, the address bus, the signals RAS and 
CAS and other signals /W and /OE, both of which are 
not shown in the drawing, are connected to each of 
1 0 DRAM chips of 16 MBit with the X4 structure. 

Data is connected to each of the chips, and is 
written therein at a time through a 40-bit data bus. 

The addresses stored in t he DRAM are described 
below. 

The addresses for writing in the DRAM are pre- 
viously sat 

As shown in Fig. 8, a 16-M word address space 
(1 word = 1 0 bits) comprising ROW addresses 00H to 
1000H and CAS addresses 00 H to 400H. 

The image data for one image plane is stored in 
the region A shown in Fig. 8. 

It is thus assumed that the ROW address thus 
ranges from 00H to 200H, and CAS address ranges 
from 00Hto200H. 

The addresses are set in the address registers 
1 03 and 1 04 shown in Fig. 2. If it is desired to further 
read an image, addressing is made within a region 
which is not superposed on the region A shown in Fig. 
8, for example, the region B shown in Fig. 8. The read 
data may be exposed and then transferred to the 
DRAM. In this embodiment, the object image and the 
dark current are stored in the regions Aand B, respec- 
tively. 

Data read from the solid state image sensor is 
the same as that in the first embodi ment. The refresh 
operation of the DRAM is also controlled to be made 
during t he time the signal HGATE Is at the LOW level 
in the same way as in the first embodiment 

The operation of storing data in the DRAM is de- 
scribed below. 

The data read from the solid state image sensor 
is quantized with 1 0 bits per pixel. Thus, the A/D con- 
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verier 5 also has two channels each having 10 bits. 

The operation is described below with reference 
to Fig. 9. 

Data is output from the solid state image sensor 
pixel by pixel for each of the channels at the same 
transfer speed as that of clock CLK2 302. However, 
the phase need not be the same as that of the dock 
CLK2. In Fig. 9, reference numeral 300 denotes the 
A/D output of one of the two channels, and reference 
numeral 301 denotes the A/D output of the other 
channel. 

The data is successively latched by the A/D reg- 
ister 123 synchronously with the clock timing of the 
clock CLK2 302, as shown by DATA 00 (304) and 
DATA 01 (305), DATA 1 0 and DATA 1 1 , DATA 20 and 
DATA 21, etc. in Fig. 9. 

The data 304 of the A/D register 123 is further 
latched by the RAM buffer 117 with timing shown by 
b in signal 309 synchronously with the dock CLK1 
303 to output data 306. On the other hand, the data 
305 of the A/D register 123 is latched by the RAM buf- 
fer 117 with timing shown by d in the signal 309 syn- 
chronously with the dock CLK1 303 to output data 
307. In this embodiment, the RAM buffer 117 can 
latch data of 40 bits. 

Data for four pixels (40 bits) is thus stored in the 
RAM buffer 1 17. The data is transferred to the output 
buffer 120 with the timing shown by e in the signal 
309, and then written to the DRAM. At this time, the 
contents of the address counter 1 07 are updated un- 
der control by the controller 116 shown in Fig. 2, and 
the address for write on the DRAM is also updated. 

When the value of the address counter 107 
agrees with the content of the address end register, 
an agreement signal is transmitted from the compar- 
ator 108 to the controller 116, and the mode of write 
in the DRAM is terminated. 

If it is desirsd to further read an image, address- 
ing is made within a region which is not superposed 
on the region A shown in Fig. 8, for example, the re- 
gion B, and the read data may be exposed and then 
transferred to the DRAM. 

The data stored in the DRAM by the above meth- 
od is transferred to the storage medium such as the 
hard disk or the like. Although this date transfer is de- 
scribed below, it is briefly described because ft is not 
related directly to the present invention. 

In this case, since data is read from the DRAM, 
t he read start address is set in the register 1 01 , and 
the read ending address is set in the register 1 02. 

Only the region A in which data was previously 
stored may be specified for transferring data of only 
one image, or both regions Aand B may be specified 
for transferring data for two images at one time. 

When date of both regions A and B are transfer- 
red, ROW address 00H and CAS address 00H are set 
in the register 1 01 , and ROW address 400H and CAS 
address 200H are set in the register 102. The mode 



for transferring data to the hard disk may be set in the 
command register 110. 

Since the hard disk is generally a 16-bit data bus 
(AT-BUS interface), the width of the data read from 

5 the DRAM is converted by the bit width converter 
121. In the bit width converter 121, the data of the 
RAM buffer 117 which is read twice from the DRAM 
for 40 bits at a t i me is stored to 80 bits, and is divided 
into five 1 6-bit port ions and output to t he hard disk in- 
to terface HDD l/F 122. At the same time, the value of 
the address counter 105 is continuously updated. 

The above operation is repeated until the content 
of the address counter agrees with the value of the 
register 1 02. When t he content oft he address cou nter 

15 1 05 agrees with t he value of the register 1 02, the sig- 
nal output from the comparator 1 06 Is transmitted to 
the controller 116, and the mode for writing data to 
the hard disk is terminated. 

The operation of the second embodiment is de- 

20 scribed below with reference to the flowcharts shown 
in Figs. 10 and 11. 

An image pickup trigger is checked by the switch 
11 shown in Fig. 1 in Step S1 . 

If the switch 11 is pressed (turned on), the menv 

25 ory is examined in Step S2 for a space capacity suf- 
ficient for one image plane. 

If there is a space capacity, the address region 
(for example, the region A shown in Fig. 8) of data for 
one image plane is set in Step S3. Image pickup 

so (charge is stored in the solid state image sensor by 
controlling the shutter and the diaphragm) is per- 
formed in Step S4. The data of the solid state image 
sensor is transferred to the memory in Step S5. The 
"image pickup ending flag" is set in Step S6 for tem- 

35 porarily storing that the object image is recorded. 

The flow then returns to Step S1. If the image 
pickup is not triggered, the image pickup ending flag 
is checked in Step S7. If the i mage pickup ending flag 
is set, dark current recording and subtraction proc- 

40 essing are performed in Steps 8 to 12. In Step S8, the 
image pickup ending flag which is set in Step S6 is 
cleared. A new address is set in a region (for example 
the region B shown in Fig. 8) which Is not superposed 
on the previous image pickup data in Step S9. Charge 

45 is stored in the solid state image sensor while the 
shutter is closed in Step S10, and the stored data is 
transferred to the memory in Step S 1 1 . The dark cur- 
rent data is subtracted from the object image data in 
Step S12. The result of subtraction is again stored in 

so the memory. This will be described in further detail 
below. 

It is checked in Step S1 3 whether of not untrans- 
f erred data remains, rf the untransferred data re- 
mains, addresses for date having a predetermined 
55 capacity are set in Step S14 (set to an intermediate 
point of t he region Ashown in Fig. 8, for example RAS 
address 100H and CAS address 200H). A predeter- 
mined amount of date is transferred from the memory 
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to the hard disk in Step S8, and the flow then returns 
to Step S1 . If the image pickup trigger is turned off, 
the flow moves to Step S7. 

Since, in Step S7, the image pickup ending flag 
is cleared in Step S8, the flow moves to Step S13. 5 
Since all data is not completely transferred in Step 

51 3, addresses are continued from the address (RAS 
address 101 H and CAS address 00H) of the portion 
from which data was previously transferred in Step 

514. The data is transferred to the hard disk in Step 10 

51 5. and the flow returns to Step S1 . When the data 
is completely transferred through Steps S1, S7, S1 3, 
S14 and S15, record of one image is completed. 

The flowchart of t he case where the result differ- 
ent from that described above is obtained by decision 1 5 
in each of the decision steps (Steps S1, S2, S7 and 
S13) is not described below for the sake of simplifi- 
cation. 

As described above, the operation can be per- 
formed by the control section 9 in accordance with 20 
the sequence shown by the flowcharts of Figs. 10 
and 11 so that the object image is pickup and stored 
in the memory, and the result of subtraction of the 
dark current from the object image is transferred to 
the hard disk. 25 

The processing in Step S12 shown in Fig. 11 is 
described in detail below. Fig. 12 is a block diagram 
illustrating details of the register 117, the operation 
section 118 and the peripheral portion thereof, which 
are shown in Fig. 3. In Fig. 12, a section 117 shown so 
by a broken line corresponds to the register 117 
shown in Fig. 3, and a section 118 shown by a broken 
line corresponds to the operation section 11 & Por- 
tions 113, 119, 120 and 123 also correspond to the 
portions denoted by the same reference numerals in 35 
Fig. 2. Reference numeral 515 denotes a buffer, and 
reference numeral 514 denotes an input data selector 
for the buffer 515 which selects any one of the A/D 
register output 123, the output of the input buffer 119 
from the DRAM or the RAM bus data 1 24 and outputs 40 
the selected data to the buffer 51 5. Reference numer- 
al 516 denotes a selector of output data from the buf- 
fer 515, which selector outputs the output data from 
the buffer 515 to one of buffer 120 or the RAM bus 
1 24. Reference numeral 51 7 denotes a selection sig- 45 
nal for selecting input of the data selector 51 4, which 
signal Is controlled by the controller 116 shown In Fig. 
2. 

Reference numeral 518 denotes a select ton sig- 
nal for selecting the output from the data selector 50 

516, which signal is controlled by the controller 116. 
Reference numeral 500 denotes an adder for adding 
data of the constant register 11 3 to data of the RAM 
bus 124 to output the result to the data bus 501. Ref- 
erence numeral 510 denotes a latch tor holding the 55 
contents of the RAM bus 124; reference numeral 506, 

an output bus for the latch 510; and reference numer- 
al 502, a subtracter for computing a difference be- 



tween the addition result 501 and the latch 51 0 to out- 
put the computation result to the bus 503. Reference 
numeral 504 denotes a minus flag signal which is 
made the HIGH level when the results of addition and 
subtraction are negative; reference numeral 505, an 
OVERFLOW flag signal which is made the HIGH level 
when the results of addition and subtraction over- 
flow; reference numeral 507, a register for holding a 
predetermined value; reference numeral 511, a com- 
parison computer for comparing the data of the RAM 
bus with the data held of the register 507, and making 
the BRIGHT signal 508 the HIGH level if the data of 
the RAM bus is greater than the predetermined val- 
ue. 

Reference numeral 509 denotes an OR gate; ref- 
erence numeral 512, OR gate output; and reference 
numeral 51 3, a latch for holding the results of addition 
and subtraction. The content of the latch 51 3 is set to 
a predetermined value (for example, 3FF [HEX]) 
when the OR gate output is the HIGH level, and is 
cleared to zero when the minus flag 504 is the HIGH 
level. 

Fig. 13 is a flowchart showing the processing op- 
eration. The operation below is mainly performed by 
the controller 116 shown In Fig. 2. 

The memory address regions in which original 
data and dark current date are stored are set in Step 
S21 . The storage addresses of the dark current date, 
i.e., the start address and the end address of the re- 
gion B shown in Fig. 8, are set in X_AD DRESS and 
X_E, respectively. Referring to Fig. 2, the start ad- 
dress is set to B_S in register 101 (XS), and the end 
address is set to B_END in register 102 (XE). 

The storage addresses of the original data, i.e., 
the start address and the end address of the region 
A shown in Fig. 8, are set in Y_ADDRESS and Y_E, 
respectively. Referring to Fig. 2, the start address is 
set to A_S in register (YS), and the end address is set 
to A_END in register 1 04 (YE) 

The dark current data is read from the X-AD- 
DRESS in Step S22 (DK_DT), and is held as X_DT by 
the latch 510 shown in Fig. 12. At this time, the se- 
lection sig nals 51 7 and 51 8 shown in Fig. 1 2 are con- 
trolled by the controller 1 1 6 shown in Fig. 2 so t hat the 
selector 514 selects the input buffer 119, and the se- 
lector 51 6 selects the RAM bus 1 24. 

The original data is read from Y_AD DRESS in 
Step S23 (l_DT) and then output to the RAM bus 124. 
The constant held by the register 113 is added to the 
original data (l_DT) on the RAM bus 124 by the adder 
500 in Step S24, and the addition result is output to 
the bus 501. The dark current date (DK_DT) of the 
latch 510 is subtracted from the data of the bus 510 
by the subtracter 502, and the subtraction result is 
output to the bus 503. The operation result is held by 
the latch 513. At this time, if the operation result is 
zero or less, the minus flag becomes t he HIGH level, 
and the latch 513 is cleared to zero. 
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If the operation result overflows, and the OVER- 
FLOW flag is the High level, or if the original data is 
greater than a predetermined value, and the BRIGHT 
signed 508 is the HIGH level, the output 512 of the OR 
gate 509 become the HIGH level, and the latch 513 
is set to a predetermined value (3 FF). 

The selection signals 517 and 518 shown in Fig. 
12 are controlled by the controller 116 shown in Fig. 
2 so that the selector 514 selects the RAM bus 124, 
and the selector 516 selects the output buffer 120 In 
Step S25. The data of the latch 513 is rewritten in the 
Y.ADDRESS on the memory. Namely, the original 
data of the Y_AD DRESS on the memory is rewritten 
by the data obtained by subtraction of the dark cur- 
rent from the original data. 

The values of the original data address (Y.AD- 
DRESS) and the dark current address (X J AT>- 
DRESS) are incremented for processing the data at 
the next address on the memory in Step S26. If the 
incremented address values are different from the 
end address (A^END) of the region A shown in Fig. 8 
in Step S27, the flow returns to Step S22 in which the 
data of the updated address Is processed by the 
same method as that described above. If X_AD- 
DRESS is not equal to A_END in Step S27, it is de- 
cided that operation is completed for the all pixels of 
one image plane. 

The above operation is applied to the case where 
noise enters the dark current image, as shown in 
Figs. 14(a) - (g). Since a predetermined value is first 
added to the original data of Fig. 14 (a), the original 
date Fig. 14(a) is offset as shown by Fig. 14(e). If 
noise as shown by Fig. 14(b) is subtracted from the 
data shown in Fig. 14(e), data as shown in Fig. 14(f) 
is obtained. After the data shown by Fig. 14(f) is 
passed through a low pass filter, the noise is offset 
to leave the original data shown by Fig. 14(g). Al- 
though, in the above embodiment, the date obtained 
by subtracting the dark current data is rewritten in the 
memory region where the original data Is previously 
stored, the date may be written in another region. In 
this case, additional address registers ZS and ZE 
may be provided In addition to the address registers 
XS, XE, YS and YE respectively shown by reference 
numerals 101 to 104 in Fig. 2. The start and end ad- 
dresses of an address region in which the data after 
subtraction is stored may be set in the registers ZS 
and ZE. Accordingly, a pair of steps for addressing 
and updating may be added to the operation shown 
in Fig. 13. 

Although, in the above embodiments, the pixel 
date of the solid state image sensor is quantized with 
10 bits, the number of bits may be set to any desired 
value and an optimum value for the system. In this 
case, the date bus width of each of the sect tons may 
be changed, and the gist of the present invention can 
be realized, as in the above embodiments. In addi- 
tion, the dark current may be stored in the memory 



before the object image is photographed. In the case 
of a continuous pickup mode, the dark current may 
be stored after continuous image pickup. In any case, 
the dark current may be stored with optimum timing 

5 for the system. 

A third embodiment of the present invention is 
described below. 

The third embodiment has the same configura- 
tion as that of each of t he above embod i ments shown 

10 in Figs. 1 , 2 and 3, and is characterized by the flow- 
chart shown in Fig. 15. 

When data for several images is transferred to 
the hard disk at a time, new date cannot be read from 
the solid state image sensor until transfer is complet- 
es ed. Thus, a predetermined amount of date may be 
transferred to the herd disk at a time. An example in 
which an image is photographed, and a next image is 
immediately photographed is described below with 
reference to Fig. 15. 

20 The image pickup trigger is checked by the 
switch 11 shown in Fig. 1 in Step S31 . If the switch is 
pushed (turned on), the memory is checked for a 
space capacity sufficient to photograph one image 
plane in Step S32. If there is a space capacity, an ad- 

25 dress region (for example, the region A shown in Fig. 
8) of data for one I mage plane Is set in Step S33. The 
image pickup operation is performed (charge is stor- 
ed in the solid state image sensor by controlling the 
shutter and the diaphragm) in Step S34, and the date 

so of the image sensor is transferred to the memory in 
Step S35. The flow then returns to Step S31. If the 
image pickup trigger is turned on, and if there is a 
space in the memory, the image pickup operation 
(Step S33) is performed . A new address is set in a re- 

35 gion (for example, the region B shown in Fig. 8) which 
is not superposed on the previous image pickup data 
In Step S34. The pickup date is stored in Step S35. If 
the memory becomes full, the flow moves from Step 
S32 to Step S36. If the image pickup trigger is turned 

40 off, the flow moves from Step S31 to Step S36. 

The memory is checked in Step S36 to see if urv 
transterred date remains. If the untra referred data 
remains, an address for a predetermined capacity of 
data is set in Step S37 (to an intermediate portion of 

45 the region A shown in Fig. 8, for example, to RAS ad- 
dress 100H and CAS address 200H). A predeter- 
mined amount of data Is transferred from the memory 
to the hard disk in Step S38. The flow again returns 
to Step S31 . If the image pickup is triggered, and if 

50 there is a space in the memory, the image pickup op- 
eration is performed in Step S33, a new address is 
set in Step S34, and the pickup date is stored in Step 
S35. 

Since the pickup date remains in Step S36, the 
55 flow moves to Step S37. 

An address is set to be continued from the por- 
tion from which data was previously transferred (RAS 
1 01 H, CAS 00H) in Step S37. The data is transferred 
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to the hard disk in Step S38, and the flow returns to 
Step S3 1. 

If the image pickup trigger Is turned off in Step 
S31, the flow moves to Step S36. The processing In 
Steps S31 , S36, S37 and S38 is repeated unless the 5 
trigger is turned on. When all data is transferred, the 
sequence is terminated. 

if there is no space in the memory in Step S32, 
the image pickup operation is not performed, and the 
flow moves to Step S36. The data is transferred to the 10 
hard disk until a sufficient space capacity is pro- 
duced. 

As described above, the image pickup trigger is 
checked each time data is transferred, and data may 
be transferred to the memory until no space is pros- is 
ent When all data is completely transferred in Step 
S36, the operation of this embodiment is terminated. 

Although, in the embodiments, the A/D converter 
has two channels, the A/D converter may have one 
channel or two or more channels. The memory may 20 
have a structure other than the X4 structure. Name- 
ly, the number of bits of data from the A/D converter 
and the number of bits of data transferred to the 
memory may be determined according to the system 
concerned. In addition, the memory is not limited to 25 
the DRAM, other memory such as SRAM or the like 
can be used if the cost and circuit scale of the system 
can be neglected. Further, the present invention can 
be applied to a system such as HDTV which handles 
a large amount of data, and usual TV systems such so 
as NTSC and PAL The second storage medium may 
be either a hard disk or anyone of various types mem- 
ory cards including DRAM, SRAM, EEPROM and 
FLASH. The second storage medium may also be 
fixed to the body or detachable therefrom. 35 

As described above, in the first embodiment, 
since the DRAM is refreshed immediately after data 
for several lines is read from the solid state image 
sensor, read from the solid state Image sensor is not 
stopped In the course of reading of an image, thereby 40 
obtaining a good image without the remarked urv 
evenness caused by variations in the storage time of 
the solid state image sensor. 

In the second embodiment, since the dark cur- 
rant is subtracted after a predetermined value is add- 45 
ed to the original image, even if noise enters the dark 
current image, the noise is offset, and the dark cur- 
rent can be subtracted without an adverse effect on 
the original image. When a pixel of the original image 
is saturated, since a predetermined value is written at so 
a corresponding address on the memory, the saturat- 
ed pixel can easily be discriminated in reproduction 
of the recorded image. The reproduction processing 
can thus easily be realized for suppressing the color 
of a high-brightness portion. In addition, dark current 55 
data may be stored either before or after an object is 
photographed according to the system, and the se- 
quence design can be realized with little limitation. 



Since the object image data is rewritten in the same 
address region on the memory after the dark current 
is subtracted from the object I mage data, a new mem- 
ory region need not be provided, and the memory ca- 
pacity can effectively utilized. 

In the third to fifth embodiments, since the start 
and end addresses for the memory are specified, 
partial write and read can easily performed, and data 
can thus be divided and then transferred toa medium 
such as a hard disk or the like. This prevents the shut- 
ter chance from being lost by the interruption of im- 
age pickup due to data transfer to the hard disk. The 
embodiments of the present invention can be applied 
to many other solid state image sensors having dif- 
ferent pixel numbers simply by changing the address- 
ing contents for write and read. In this way, since the 
memory is appropriately controlled while being relat- 
ed to the image sensor and other peripheral devices, 
the embodiments can easily be applied to a high- 
definition camera which requires a large volume of 
data. 



Claims 

1 . An image processing apparatus comprising: 

an image sensor for sensing an optical im- 
age and converting the image into image data; 

a memory for storing storing the image 
data from said image sensor; and 

refresh signal generation means for re- 
freshing said memory at a first time interval dur- 
ing which no image data is read from said solid 
state image sensor and at a second time intervaJ 
during storing of image data from said image sen- 
sor. 

2. An image processing apparatus comprising: 

a solid state image sensor for sensing an 
optical image and converting the image Into an 
electric signal, said solid state image sensor hav- 
ing at least one pixel; 

a storage medium for A/D converting the 
electric signal from said solid state image sensor 
to A/D-converted output data and storing the A/D- 
converted output data therein; 

first output storing means for storing a first 
output of A/D conversion of a first image on said 
solid state image sensor as a first content in a 
first predetermined region of said storage me- 
dium; 

second output storing means for storing a 
second output of A/D conversion of a second im- 
age on said solid state image sensor as a second 
content in a second predetermined region of said 
storage medium; and 

operation storage means for adding a first 
predetermined value to said second content of 
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said second region of said storage medium to ob- 
tain an addition result, subtracting said first con- 
tent of said first region from the addition result to 
obtain a subtraction result, and storing the sub- 
traction result in said second region of said stor- 
age medium, 

wherein, when the A/D-converted output 
data of any desired pixel of said solid state image 
sensor in which said second image is stored is 
within a first predetermined range, a second pre- 
determined value is stored in a region of said stor- 
age medium corresponding to said desired pixel, 
and when the addition result is within a second 
predetermined range, the addition result is re- 
placed by a third predetermined value, and when 
the subtraction result is within a third predeter- 
mined range, the subtraction result is replaced by 
a fourth predetermined value. 

3. An image processing apparatus comprising: 

a solid state image sensor for converting 
an optical image into output data; 

a memory for storing the output data from 
said solid state image sensor 

first counter means for counting which is 
initialized to a first address; 

a first register, said first register holding a 
second address; 

first comparison means for comparing the 
content of said first counter with the content of 
said first register; 

second counter means for counting which 
is initialized to a third address; 

a second register, said second register 
holding a fourth address; 

second comparison means for comparing 
the content of said second counter with the con- 
tent of said second register; 

address selection means for selecting one 
of the content of said first counter means and the 
content of said second counter means and out- 
putting t he selected content to said memory; and 

memory control circuit means for control- 
ling the operation of said memory. 

4. An image processing apparatus according to 
Claim 3, wherein said first address is the start ad- 
dress for write in said memory, said second ad- 
dress is the end address for write, said third ad- 
dress is the start address for read from said 
memory, said fourth address is the end address 
for read, and said output data being read from 
said sol id state image sensor is stopped based on 
an output from said first comparison means or 
said second comparison means. 

5. An image processing apparatus according to 
Claim 4, further comprising: 



storage means for storing memory data of 
said memory; 

access means for continuously accessing 
any desired region of said memory; and 
5 transfer means for transferring in blocks, 

to said storage means, the memory data in any 
desired region of said memory having a capacity 
smaller than that of a single image plane of said 
solid state image sensor so that the output data 
10 is transferred from said solid state image sensor 
to said memory between transfer of respective 
said blocks. 

6. An apparatus comprising a charge storage image 
15 sensorand memory means including memory re- 
fresh control means whose operation sequence 
depends on whether image data is being read 
from the image sensor, whereby image read ti me 
is not reduced for memory refresh operations. 

20 

7. An apparatus comprising an image sensor and 
image memory means, wherein an object image 
and a dark current image can be combined in the 
memory means such that noise in the dark cur- 

25 rent image can be compensated in the output im- 
age. 

8. An electric camera including an image sensor 
and memory means for storing image date syn- 
chronously with the reading of such data from the 
image sensor, wherein the memory means com- 
prises dynamic random access memory and/or 
wherein a memory controller effects a variable 
mapping between the image sensor elements 
and memory locations. 
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